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THE AUTHOR STATES THAT DETERMINING THE CRITERIA RELEVANT 
TO THE EDUCATION PROCESS IS EQUIVALENT TO DEFINING OBJECTIVES 
OF HIGHER EDUCATIC-N. THE TRADITIONAL APPROACH TO DEFINING 
CRITERIA HAS BEEN TO SPECIFY FRCM CONCEPTUAL CRITERIA 
(EDUCATIONAL OBJECTIVES), THE CRITERICM PERFORMANCE. AN 
IMPROVED APPROACH USES A CLASSIFICATION SCHEME TO ORGANIZE 
THE GLOBAL CONTENT OF ABSTRACT STATEMENTS OF EDUCATIONAL 
GOALS INTO AREAS OF RESEARCH INTEREST SO THAT CRITERION 
PERFORMANCES CAN BE SPECIFIED WITHIN THE CONTEXT CF A 
PARTICULAR STUDY OF RESEARCH PROGRAMS. A SIMPLE EXAMPLE OF 
SUCH A CLASSIFICATION WOULD INCLUDE COGNITIVE OUTCOMES SUCH 
AS STUDENTS* KNCWLEDGE AND AFFECTIVE COTCOMES SUCH AS 
MOTIVATION AND VALUES. THE CRITERIOInI DEFINITICN SHOULD MAKE 
SENSE WITH regard TO THE PROBLEM BEING INVESTIGATED AND TO 
THE POSSIBLE APPLICATIONS OF THE FINDINGS. THE AUTHOR FURTHER 
STATES THAT ALL RESEARCH PROGRAMS ShKXJLD BE FLEXIBLE ENOUGH 
TO INCLUDE, IF NECESSARY, EDUCATIONAL OUTCOt-lES NOT CONSIDERED 
IN THE ORIGINAL DESIGN. TWO-BY-TWO CLASS IF I CAT ICN CF A 
FRESHMAN INPUT SURVEY INCLUDED — (1) ASSESSMENT CF 
INTELLECTUAL OUTCOMES SUCH AS GRADE POINT AVERAGES, (2) 
EVALUATION CF CHANGES IN THE AREAS CF STUDENT VALUES, 
ATTITUDES, AND PERSCNALITY, (3) THE PROCESS CF VOCATIONAL 
CHOICE, AND (4) LASTING OUTCOMES OF STUDENT BEHAVIOR. THIS 
PAPER WAS PRESENTED AT THE SYMPOSIUM, *’ IMPLICATIONS OF A 
PROGRAM CF RESEARCH CN STUDENT DEVELOPMENT IN HIGHER 
EDUCATION,*’ AMERICAN PERSONNEL AND GUIDANCE ASSOCIATION 
MEETINGS (DALLAS, MARCH 21, 1967). (PS) 



ED Oil 392 

CRITERIA ON STUDENT DEVELOPMENT. 

BY- PANOS, ROBERT J. 

ECRS PRICE MF-$0.ri9 HC-$0.52 13P. 



O 

ERIC 



ED011392 




U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFFICE OF EDUCATION 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POUCY. 



Criteria on Student Development* 
Robert J. Panos 



American Council on Education 



*Presented as part of a symposium. Implications of a Program of Research 
on Student Development In Higher Education. American Personnel and Guidance • 
Association Meetings, Dallas, Texas, March 21, 1967. 



CG 000 Q88 



o 

ERIC 















Criteria on Student Development 
Robert J. Panos 

9 

American Council on Education 
• • 

A basic concern of any research program concerned with the outcomes 
of higher education is to assess student behaviors that are relevant to the 
goals of the educational enterprise and to relate these observations to 
the environmental influences that characterize the educational intervention. 
Since this area of concern involves extensive (and . expensive) longitudinal 
research, there have been few studies of a comprehensive and representa- 
tive nature. In addition, there are a number of difficult problems in 
the general area which remain largely unsolved. These larger problems 
can be subsumed under three broad — but clearly not independent — categories. 
First, there is the problem of definition. What are the relevant criteria? 
Second, there is the question of measurement. What observations of stu- 
dent behavior, in what social context, best elicit the r^elevant data? 

Third, there is the problem of research design, ^at methodology of in- 
ferential proced|ires will most adequately organize and display the inter- 

% 

k 

relations among the original observations in an intellectually satisfying 
and objectively convincing manner? 

The more difficult problems of design and methodology — with regard 
to their bearing on the Council’s research program — have been discussed 
elsewhere (Astin, Panos, and Creager, 1966). The problem of definition 
is spmewhat less difficult because, being primarily analytical, it is 
more speculative than it is real. For this reason, the major focus of 
this paper will be at least correspondingly hypothetical. 
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Asking which criteria are relevant to the process of education is 

equivalent to asking what are the objectives of higher education. More 

specifically, what are the particular student behaviors that the various 

• • 

educational interventions are intended to influence? However, going from 
"relevant criteria" to the "objectives of higher education" is by no means 
a simplifying reduction. There are at least two approaches one can utilize 
in attempting to come to grips with this definitional problem. One way 
Is to specify, from the statement of an educational objective, the opera- 
tional (that is, the behavioral) manifestations of the outcome of interest 
and the social context in which that behavior is supposed to occur. This 
I statement and specification of the objective is the conceptual criterion 
(Astin, 1964). A criterion performance , then, can be conceived as any 
j transactional event between an individual and his environment that is 
judged to be relevant to the conceptual criterion. Observations of the 
criterion performance, of themselves or after statistical manipulation, 
become criterion measures. This, of course, is the traditional evalua- 
tlon process. 

Educational objectives, however, because they are derived from a 
heterogeneous variety of sources — for example, administrators, teachers, 
students, and subject-matter specialists — and because they develop gradu- 
ally, are initially poorly defined and sometimes apparently contradictory. 
Unfortunately, they usually remain that way. Thus, although the ideal 
source for ascertaining the relevant criteria of an educational interven- 
tion would be to determine them from statements of educational objectives, 
these statements — taken at the level of the system of higher education — 
are too global and abstract to function as a source of viable conceptual 
criteria. 
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The fact is that educational goals, as presented in college catalogs 
for example, consist largely of superficial and essentially nonfunctional 
statements about the development of students who value the "intellectual" 
way of life and the like. Although it is obvious that most persons would 
agree that such goals are relevant and desirable, it is also obyious that 
the words in such statements connote different meanings to different indi- 
viduals. The task of translating such nonfunctional statements into con- 
ceptual criteria, that is, into specifiable operations, is very nearly im- 
. possible. We believe that effort in this area is better devoted to ob- 
taining a clear picture of what is actually happening to the students. 
Perhaps, when we have been able to discover and adequately document what 
the outcomes of college are, we can wondei about whether or. not we like 
what they are and what we can or cannot do about them. 

This brings us to a second approach for specifying ,the outcomes 
of higher education. In this approach, the global content of abstract 
statements of educational goals are arrayed under similarly global and 
abstract labels into areas of research Interest. Utilizing such a classi- 
fication scheme, it is then possible to specify criterion performances 
within the context of a particular study or research program. Figure 1 
displays the results of one such attempt to sort out relevant sources of 
criteria on student development in terms of a simple 2x2 classification. 

In Figure 1, the so-called behavioral domain has been divided into 
the now traditional categories of cognitive or intellective outcomes, and 
nonlntellective or affective outcomes. The cognitive domain includes such 
outcomes as the student’s knowledge, abilities, and intelligence, while 
affective outcomes include his motivation, values, and attitudes. 
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Criterion measures can be characterized, according to their source 
Into those observations which, in themselves, are in the form of measures, 
and observations that imply a measure of some hypothetical latent (psycho- 
logical) construct. Empirically verifiable or observable outcomes include 
such measures as whether or not a student obtained a terminal degree and 
at what level, his extracurricular achievements and awards, and certain 
aspects of his overt behavior toward his fellow man. Measures of nonob- 
servable outcomes, which can only be inferred and on which an individual’s 
position on an hypothetical latent continuum can only be estimated, include 
such personality "traits" as intelligence, .values, and attitudes. The cru- 
cial methodological differentiation in this classification is between directly 
observable events and outcomes which are not completely revealed by any set 
of observable indicators. 

Although this second approach apparently ignores important poten- 
tial interactions — for example, between the cognitive and affective do- 
mains, or between an individual’s aptitude, values, interpersonal relations, 
and attained achievements — the classification is not intended as a repre- 
sentation of reality, but rather as an heuristic tool. Its function is 
• to force us to recognize explicitly those criteria that we have included 
as part of our research design, and — perhaps more importantly — to recog- 
nize potentially relevant behavioral outcomes that the design has excluded, 
or that at least are not being explicitly dealt with in a particular study 
or research program. 

It is essential to note that regardless of their source, conceptual 
criteria (educational objectives) are not subject to empirical verification 
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or tests. They are rational statements of desired behavioral or social 
outcomes. In short, conceptual criteria are representations of normative 
assumptions about the nature of man, the nature of knowledge, and the 
nature of reality. Thus, educational criteria can be accepted or rejected 
only on rational grounds. Furthermore, the relevance or lack of relevance 
of the designated criterion performance is similarly judgmental; that is, 
relevance is not empirically testable. 

It should be clear from the preceding discussion that, in our 
view, criteria cannot be empirically validated. Thus, the only method for 
judging the relevance of a criterion measvre to an educational outcome is 
on the basis of a rational analysis. The point, of course, is that the 
consumer or potential user of the findings of any reasearch that has im- 
plications for educational practice has a special responsibility not only 

to scrutinize the research design and methodology that produced the findings, 

: / 

but also to determine the relevance of the educational outcome, as defined 
In the research, to his immediate concerns. In other words, the criterion 
definition should make sense with regard both to the problem being inves- 
tigated and to the possible applications of the findings. 

Because of the time needed for the conduct of most educational re- 
search, it is important that the design of research projects attempt to 
Incorporate those educational objectives that are relevant to the ongoing 
educational process, and that the research program be flexible enough to 
Include, at any subsequent stage, educational outcomes that are not being 
considered in the original design. In this regard, collaboration and com- 
munication among all persons interested in the study of higher education 
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would seem more crucial during these early planning stages, but is clearly 
necessary throughout the conduct of the research. 

Obviously, the problem of which student outcomes are relevant to 
the goals of higher education and how they can best be evaluated is far 
from being solved. It is just as obvious that general and abstract dis- 
cussions (such as this one), while they may be useful, will not solve the 



problem. It might be useful, however, to briefly indicate the types of 
Items we have included in our freshmen input surveys in an attempt to de- 
fine a reference point in time to which the subsequent behavior of the 
student can be related. 

In the box labeled A in Figure 1, we have collected student input 
data relevaii!: to assessing such outcomes as overall college grade point 
average, grade point average in major field of study, and subsequent 
achievement test performances such as performance on the area tests of 
the Graduate Record Examination. Although the prediction of these kinds 
of intellective outcomes has a history of more than fifty years of research, 
the predictability of academic achievement remains at a level far less than 
that theoretically attainable. 

In the box labeled B, we hope to evaluate changes in the areas of 
student values, attitudes, personality, and educational aspirations. We 
have already collected input information concerning the student *s self-con- 
cept through a trait self-rating technique, and information concerning his 
values-orientation through a rating scale of life goals. We have attempted 



to include in these instruments items that can be conceptually referred 
to Interpersonal and noninterpersonal behaviors. By repeating these items 
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in follow-up studies, we will be able to evaluate changes over time and 
to relate these observations to the college experience. In the area of 
attitudes, we have included input items in the freshmen survey that V7ill 
permit us to study hov7 the college environment shapes the student’s per- 
ceptlon of and attitude toward his college. 

In box C, we hope to shed some light on the process of vocational 
choice. For example, Astln (1965) — based on a theory of selective envir- 
onmental reinforcement — has suggested the hypothesis that an individual’s 

development during college may be largely determined by the types of in- 
• ✓ 

terpersonal peer relationships available to him on the campus. Thus, the 
student’s career choice tends to shift in the direction of the dominant 
or modal choice of his fellow students. We also hope to evaluate outcomes 
such as relatively high-level extracurricular achievements including awards 
In the arts and sciences, level of final degree attained, and the personal 
and environmental factors that are associated with persistence in college 
Including the later vocational development of the dropout. 

Box D presents the most difficult problem because it is here that 
outcomes of a lasting nature or relatively permanent changes in the stu- 
dent’s behavior vls-a-vls society at large are arrayed. That is, these 
types of outcomes arc usually not manifest in the Individual’s behavior 
until well after he has left college. We have Included a large array of 
Input items in the freshmen survey for a controlled evaluation of what may 
be called day-to-day Interpersonal and noninterpersonal behaviors. Although 
some of these items — such as gambled, drank beer, participated in organi- 
zed demonstrations, smoked cigarettes, or, for that matter, prayed — may not 
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be the kinds of behavior a college curriculum is explicitly attempting to 
Influence, they arc, nevertheless, possible outcomes of the college experi- 
ence* In short, in all of our studies, we should not lose sight of the 
fact that educational Interventions have a variety of consequences, and 
that unplanned for outcomes— including "side effects" of education — arc 
necessarily part of the educational experience. 

In our research, we begin v/ith the assumption that there is no one 

/ 

method for effecting change in student behavior* We are, nevertheless, 
primarily concerned with examining how changes in student behavior come 
about* Thus, our focus is on the process of development and the identi- 
fication and Isolation of critical variables* We believe that extensive 
longitudinal research of a comprehensive nature is necessary in order to 
make finer distinctions among students and their college experiences which 
may be of use to institutions, guidance personnel, and Instructors in map- 
ping out a strategy of learning for their particular students* In short, 
we believe that a thorough knowledge about the student, his environment 
for learning, and his development during the college years, will help us 
to discover which educational objectives — implicit or explicit are actually 
being achieved, and what can or cannot be done about them* 
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Figure 1. Scheme for classifying types of criteria on student development. 
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